
United States Patent and Trademark Office 



UNITED STATES DEPARTMENT OF COMMERCE 
United States Patent and Trademark Office 
Address: COMMISSIONER FOR PATENTS 
P.O. Box 1450 

Alexandria, Virginia 22313-1450 
www.uspto.gov 



APPLICATION NO. j 


FILING DATE 


FIRST NAMED INVENTOR 


| ATTORNEY DOCKET NO. 


CONFIRMATION NO. 


10/826,159 


04/15/2004 


Hua-Jun Zeng 


MS1-1920US 


8623 



22801 7590 04/24/2007 

LEE & HAYES PLLC 

421 W RIVERSIDE AVENUE SUITE 500 

SPOKANE, WA 99201 



EXAMINER 



SANDERS, AARON J 



ART UNIT 



PAPER NUMBER 



2168 



SHORTENED STATUTORY PERIOD OF RESPONSE 



NOTIFICATION DATE 



DELIVERY MODE 



3 MONTHS 04/24/2007 ELECTRONIC 

Please find below and/or attached an Office communication concerning this application or proceeding. 

If NO period for reply is specified above, the maximum statutory period will apply and will expire 6 MONTHS 
from the mailing date of this communication. 

Notice of this Office communication was sent electronically on the above-indicated "Notification Date" and 
has a shortened statutory period for reply of 3 MONTHS from 04/24/2007. 

Notice of the Office communication was sent electronically on above-indicated "Notification Date" to the 
following e-mail address(es): 

lhptpms@leehayes.com 



PTOL-90A (Rev. 10/06) 



Office Action Summary 


Application No. 

10/826,159 


Applicant(s) 

ZENG ET AL. 


examiner 

Aaron J. Sanders 


Art Unit 
2168 





- The MAILING DA TE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDON ED (35 U.S.C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )E3 Responsive to communication(s) filed on 02 February 2007 . 
2a)E3 This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) [3 Claim(s) 1-40 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) ^ Claim(s) 1-10,12-31 and 33-40 is/are rejected. 

7) [3 Claim(s) 1 1 and 32 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) Q The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)Q Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachment(s) 

1) ^ Notice of References Cited (PTO-892) 

2) O Notice of Draftsperson's Patent Drawing Review (PTO-948) 

3) ^ Information Disclosure Statement(s) (PTO/SB/08) 

Paper No(s)/Mail Date 12/12/2006 and 02/02X007 . 



4) □ Interview Summary (PTO-413) 

Paper No(s)/Mail Date. . 

5) □ Notice of Informal Patent .Application 

6) □ Other: . 



U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) 



Office Action Summary 



Part of Paper No./Mail Date 20070330 



Application/Control Number: 1 0/826, 1 59 Page 2 

Art Unit: 2168 

DETAILED ACTION 
Response to Amendment 

This Office action has been issued in response to amendment filed 2 February 2007. 
Claims 1-40 are pending. Applicant's arguments have been carefully and respectfully 
considered, and some are persuasive, while others are not. Accordingly, objections and 
rejections have been removed where arguments were persuasive, but rejections have been 
maintained where arguments were not persuasive. Accordingly, claims 1-10, 12-31, and 33-40 
are rejected, claims 1 1 and 32 are objected to, and this action has been made FINAL, as 
necessitated by amendment. 

Claim Rejections - 35 USC § 112 Second Paragraph 

The following is a quotation of the second paragraph of 35 U.S.C. 1 12: 

The specification shall conclude with one or more claims particularly pointing out and distinctly claiming the 
subject matter which the applicant regards as his invention. 

Claims 1, 13, 22, and 34 are rejected under 35 U.S.C. 1 12, second paragraph, as being 
indefinite for failing to particularly point out and distinctly claim the subject matter which 
applicant regards as the invention. 

Claims 1,13, 22, and 34 provide for the use of "reinforced clusters", but, since the claim 
does not set forth any steps involved in the method of responding to a search query and 
suggesting search terms relevant to the query, it is unclear what method applicant is intending to 
encompass. A claim is indefinite where it merely recites a use without any active, positive steps 
delimiting how this use is actually practiced. 
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Claim Rejections - 35 USC §101 

35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

Claims 1-11, 13-20, 22-32, and 34-39 are rejected under 35 U.S.C. 101 because the 
claimed invention is directed to non-statutory subject matter. 

Claims 1-11 are directed to a method. Claims 13-20 are directed to a device. Claims 22- 
32 are directed to a storage medium. Claims 34-39 are directed to a device. The claimed subject 
matter lacks a practical application of a judicial- exception (law of nature, abstract idea, naturally 
occurring article/phenomena) since it fails to produce a useful, concrete, and tangible result. 

Specifically, the claimed subject matter does not produce a useful result because the 
claimed subject matter fails to sufficiently reflect at least one practical utility set forth in the 
descriptive portion of the specification. More specifically, while the described practical utility 
(utilities) is (are) directed to search term suggestion via reinforced clustering of multi-type data 
objects, the claimed subject matter relates ONLY to "iteratively clustering the multi-type data 
objects". The claimed recitation of a "use", without setting forth any steps involved in the 
method, results in an improper definition of a method, i.e., results in a claim which is not a 
proper method claim under 35 U.S.C. 101. See for example Ex parte Dunki, 153 USPQ 678 
(Bd.App. 1967) and Clinical Products, Ltd v. Brenner, 255 F. Supp. 131, 149 USPQ 475 
(D.D.C. 1966). 

Further, the claimed subject matter does not produce a tangible result because the claimed 
subject matter fails to produce a result that is limited to having real world value rather than a 
result that may be interpreted to be abstract in nature as, for example, a thought, a computation, 
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or manipulation of data. More specifically, the claimed subject matter provides for "iteratively 
clustering the multi-type data objects". This produced result remains in the abstract and, thus, 
fails to achieve the required status of having real world value. 

As per claims 34-39, the device does not require any hardware, making it software per se. 
As such, the instant claims are non-statutory. 

Claim Rejections - 35 USC §102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by another filed 
in the United States before the invention by the applicant for patent or (2) a patent granted on an application for 
patent by another filed in the United States before the invention by the applicant for patent, except that an 
international application filed under the treaty defined in section 351(a) shall have the effects for purposes of this 
subsection of an application filed in the United States only if the international application designated the United 
States and was published under Article 21 (2) of such treaty in the English language. 

Claims 1-10, 12-31, 33-35, 37-40 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Schuetze et al., U.S.P.G. Pub. 2003/01 10181. . 

As per claims 1-10, 12-31, 33-35, 37-40, Schuetze et al. teach: 
1. A computer-implemented method comprising: 

identifying relationships between multi-type data objects, wherein the multi-type data 
objects comprise at least one object of a first type and at least one object of a second type 
different from the first type (See e.g. [0022], "Disparate types of information such as text, image 
features and usage are referred to as 'modalities.' Multi-modal clustering hence is the grouping 
of objects that have data from several modalities associated with them" and [0033], "various 
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document features in different modalities are appropriately weighted and combined to form 
clusters representative of overall similarity"); 

iteratively clustering the multi-type data objects in view of respective ones of the 
relationships to generate reinforced clusters (See e.g. [0035], "Using the system, a user 
progressively narrows a collection to a small number of elements of interest, similar to the 
Scatter/Gather system developed for text browsing, except the Scatter/Gather method is extended 
hereby to use multi-modal features" where, see [0043], "iterative clustering and selection of 
cluster subsets can help a user identify images of interest"); and 

wherein the reinforced clusters are for use by a search term suggestion component to 
respond to a search query from a user with terms relevant to the search query (See e.g. Fig. 12 
and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms. . . The user decides to scatter the first text cluster containing the terms 'ancient' and 
'cathedral' again based on text. . . As described above, this causes the system to refine the 
existing selected cluster into smaller separate clusters"). 

2. The method of claim 1, wherein the relationships comprise inter-layer relationships 
including one or more of content related information, user interest in an associated topic, and 
user interest in an associated Web page (See e.g. [0032], "The method takes/advantage of 
multiple ways in which a user can specify items of interest. For example, in images, features 
from the text and image modalities can be used to describe the images. . . clustering may be 
performed on a different feature (e.g., surrounding text, image URL, image color histogram, 
genre of the surrounding text)"). 
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3. The method of claim 1, wherein the relationships comprise intra-layer relationships 
including one or more of query refinement(s), recommended Web page(s), and relationship(s) 
between respective users (See e.g. [0028], "It is also useful to be able to track individuals' 
information access habits by way of the characteristics of the documents those users access, 
thereby enabling a recommendation system in which users are assigned to similar clusters")- 

4. The method of claim 1, wherein each of the multi-type data objects are related to one 
or more of a search query data object type, a selected Web page type, and a user information type 
(See e.g. [0037], "Multi-modal features may take on many forms, such as user information, text 
genre, or analysis of images"). 

5. The method of claim 1, wherein respective ones of the relationships are weighted to 
indicate importance to associated objects of the multi-type data objects (See e.g. [0033], "various 
document features in different modalities are appropriately weighted and combined to form 
clusters representative of overall similarity"). 

6. The method of claim 1, wherein identifying an iteratively clustering are performed for 
search term suggestion (See e.g. Fig. 12 and [0171], "A snapshot of the screen displaying five 
returned text clusters 1216, 1218, 1220, 1222, and 1224 is shown in the left half of FIG. 12. 
These clusters are the clusters closest to the query terms... The user decides to scatter the first 
text cluster containing the terms 'ancient' and 'cathedral' again based on text. . . As described * 
above, this causes the system to refine the existing selected cluster into smaller separate 
clusters"). 

7. The method of claim 1, wherein iteratively clustering further comprises propagating 
clustering results of a first iteration to all related data objects of the multi-type data objects, at 
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least two of the related data objects being of heterogeneous data type, the propagating being used 
to enhance clustering of respective ones of the multi-type data objects in a second iteration of 
reinforced clustering operations (See e.g. [0152], "Scatter/Gather iteratively refines a search by 
'scattering' a collection into a small number of clusters, and then a user 'gathers' clusters of 
interest for scattering again. The Scatter/Gather method is extended by the invention to extend to 
a multi-modal, multi-feature method, using both text and image features to navigate a collection 
of documents with text and images"). 

8. The method of claim 1, wherein iteratively clustering further comprises determining a 
similarity between individual ones of the multi-type data objects, the similarity being a function 
of one or more of inter-object and intra-object content similarity and similarities between 
respective ones of the relationships (See e.g. [0003], "The invention relates to... an efficient 
scheme for assigning data objects in a collection to clusters based on similarities in their contents 
and characteristics"). 

9. The method of claim 1, wherein iteratively clustering further comprises merging 
related ones of the multi-type data objects to reduce feature space dimensionality of the related 
ones (See e.g. [0033], "various document features in different modalities are appropriately 
weighted and combined to form clusters representative of overall similarity"). 

10. The method of claim 1, wherein the method further comprises mutually reinforcing 
an importance of individual ones of the multi-type data objects within an object type and 
between different object types (See e.g. [0097], "The use of token, frequency weight and inverse 
context frequency weight for the embedding employed by the invention is consistent with the 
following intuitive description. Each additional occurrence of an element (or word, for example) 
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in a context (e.g., a document) reflects an increased level of importance for that element as a 
descriptive feature")- 

12. The method of claim 1, and further comprising: 

responsive to receiving a term from a user, comparing the term with a feature space of 
objects in the reinforced clusters (See e.g. Fig. 12, "Query Words" 1210); 

responsive to comparing, identifying one or more search term suggestions (See e.g. Fig. 
12 and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms"); and 

communicating the search term suggestions to the user (See e.g. Fig. 12, "Text 
Cluster[s]" 1216, 1218, 1220, 1222, and 1224). 

13. A computing device comprising: 

a processor (See e.g. Fig. 1, "processor 122"); and 

a memory coupled to the processor, the memory comprising computer-program 
instructions executable by the processor for (Fig. 1 and [0077], "the collection 120 is hosted by 
one or more servers also coupled to the network 124" where a "server" includes a "memory"): 

identifying relationships between multi-type data objects, wherein the multi-type data 
objects comprise at least one object of a first type and at least one object of a second type 
different from the first type (See e.g. [0022], "Disparate types of information such as text, image 
features and usage are referred to as 'modalities.' Multi-modal clustering hence is the grouping 
of objects that have data from several modalities associated with them" and [0033], "various 
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document features in different modalities are appropriately weighted and combined to form 
clusters representative of overall similarity"); 

iteratively clustering the multi-type data objects in view of respective ones of the 
relationships to generate reinforced clusters (See e.g. [0035], "Using the system, a user 
progressively narrows a collection to a small number of elements of interest, similar to the 
Scatter/Gather system developed for text browsing, except the Scatter/Gather method is extended 
hereby to use multi-modal features" where, see [0043], "iterative clustering and selection of 
cluster subsets can help a user identify images of interest"), each relationship of the relationships 
being weighted to indicate an importance to associated objects of the multi-type data objects (See 
e.g. [0033], "various document features in different modalities are appropriately weighted and 
combined to form clusters representative of overall similarity"); and 

wherein the reinforced clusters are for use by a search term suggestion component to 
respond to a search query from a user with terms relevant to the search query (See e.g. Fig. 12 
and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms... The user decides to scatter the first text cluster containing the terms 'ancient' and 
'cathedral' again based on text... As described above, this causes the system to refine the 
existing selected cluster into smaller separate clusters"). 

14. The computing device of claim 13, wherein the relationships comprise interVlayer 
relationships including one or more of content related information, user interest in an associated 
topic, and user interest in an associated Web page (See e.g. [0032], "The method takes advantage 
of multiple ways in which a user can specify items of interest. For example, in images, features 
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from the text and image modalities can be used to describe the images. . . clustering may be 
performed on a different feature (e.g., surrounding text, image URL, image color histogram, 
genre of the surrounding text)"). 

15. The computing device of claim 13, wherein the relationships comprise intra-layer 
relationships including one or more of query refinement(s), recommended Web page(s), and 
relationship(s) between respective users (See e.g. [0028], "It is also useful to be able to track 
individuals' information access habits by way of the characteristics of the documents those users 
access, thereby enabling a recommendation system in which users are assigned to similar 
clusters"). 

16. The computing device of claim 13, wherein identifying an iteratively clustering are 
performed for search term suggestion (See e.g. Fig. 12 and [0171], "A snapshot of the screen 
displaying five returned text clusters 1216, 1218, 1220, 1222, and 1224 is shown in the left half 
of FIG. 12. These clusters are the clusters closest to the query terms... The user decides to scatter 
the first text cluster containing the terms 'ancient' and 'cathedral' again based on text. . . As 
described above, this causes the system to refine the existing selected cluster into smaller 
separate clusters"). 

. 17. The computing device of claim 13, wherein the computer-program instructions for 
iteratively clustering further comprise instructions for aggregating data object relationships to 
related ones of the multi-type data objects based on content of the reinforced clusters (See e.g. 
[0031], "Each modality within each document is described herein by an n-dimensional vector, 
thereby facilitating quantitative analysis of the relationships among the documents in the 
collection" where, see [0076], "As illustrated in FIG. 1, each document (for example, an HTML 
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document 110) chosen from a collection 120 maps to a set of feature vectors 1 12, one for each 
modality (for example, a text vector 1 14 and a URL vector 1 16)"). 

18. The computing device of claim 13, wherein the instructions for iteratively clustering 
further comprise instructions for determining a similarity between individual ones of the multi- 
type data objects, the similarity being a function of one or more of inter-object and intra-object 
content similarity and similarities between respective ones of the relationships (See e.g. [0003], 
"The invention relates to... an efficient scheme for assigning data objects in a collection to 
clusters based on similarities in their contents and characteristics"). 

19. The computing device of claim 13, wherein the instructions for iteratively clustering 
further comprise instructions for merging related ones of the multi-type data objects to reduce 
feature space dimensionality of the related ones (See e.g. [0033], "various document features in 
different modalities are appropriately weighted and combined to form clusters representative of 
overall similarity"). 

20. The computing device of claim 13, wherein the instructions for iteratively clustering 
further comprise instructions for iteratively clustering until all object types represented by the 
multi-type data objects converge (See e.g. [0078], "the collection 120 comprises all known 
documents that will ever by [sic] processed by a system according to the invention" where the 
"process" is illustrated in Fig. 3 and where "converge" is defined in Applicant's specification 
paragraph [0074] as, "each type of the different kinds of nodes and links are examined to obtain 
structural information that can be used for clustering"). 

21. The computing device of claim 13, and further comprising instructions for: 
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responsive to receiving a term from a user, comparing the term with a feature space of 
objects in the reinforced clusters (See e.g. Fig. 12, "Query Words" 1210); 

responsive to comparing, identifying one or more search term suggestions (See e.g. Fig. 
12 and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms"); and 

communicating the search term suggestions to the user (See e.g. Fig. 12, "Text 
Clusters]" 1216, 1218, 1220, 1222, and 1224). 

22. A tangible computer-readable data storage medium comprising computer-executable 
instructions executable by a processor for: 

identifying one or more of intra-layer and inter-layer relationships between multi-type 
data objects, wherein the multi-type data objects comprise at least one object of a first type and at 
least one object of a second type different from the first type (See e.g. [0022], "Disparate types 
of information such as text, image features and usage are referred to as 'modalities.' Multi-modal 
clustering hence is the grouping of objects that have data from several modalities associated with 
them" and [0033], "various document features in different modalities are appropriately weighted 
and combined to form clusters representative of overall similarity"); 

iteratively clustering the multi-type data objects in view of respective ones of the 
relationships to generate reinforced clusters (See e.g. [0035], "Using the system, a user 
progressively narrows a collection to a small number of elements of interest, similar to the 
Scatter/Gather system developed for text browsing, except the Scatter/Gather method is extended 
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hereby to use multi-modal features" where, see [0043], "iterative clustering and selection of 
cluster subsets can help a user identify images of interest"); and 

wherin the reinforced clusters are for use by a search term suggestion component to 
respond to a search query from a user with terms relevant to the search query (See e.g. Fig. 12 
and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms. . . The user decides to scatter the first text cluster containing the terms 'ancient' and 
'cathedral' again based on text. . . As described above, this causes the system to refine the 
existing selected cluster into smaller separate clusters"). 

23. The computer-readable medium of claim 22, wherein the inter-layer relationships 
comprise one or more of content related information, user interest in an associated topic, and 
user interest in an associated Web page (See e.g. [0032], "The method takes advantage of 
multiple ways in which a user can specify items of interest. For example, in images, features 
from the text and image modalities can be used to describe the images. . . clustering may be 
performed on a different feature (e.g., surrounding text, image URL, image color histogram, 
genre of the surrounding text)"). 

24. The computer-readable medium of claim 22, wherein the intra-layer relationships 
comprise at least one of query refinement(s), recommended Web page(s), and relationship(s) 
between respective users (See e.g. [0028], "It is also useful to be able to track individuals' 
information access habits by way of the characteristics of the documents those users access, 
thereby enabling a recommendation system in which users are assigned to similar clusters"). 
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25. The computer-readable medium of claim 22, wherein each of the multi-type data 
objects are related to at least one of a search query data object type, a selected Web page type, 
and a user information type (See e.g. [0037], "Multi-modal features may take on many forms, 
such as user information, text genre, or analysis of images"). 

26. The computer-readable medium of claim 22, wherein respective ones of the 
relationships are weighted to indicate an importance to associated objects of the multi-type data 
objects (See e.g. [0033], "various document features in different modalities are appropriately 
weighted and combined to form clusters representative of overall similarity"). 

27. The computer-readable medium of claim 22, wherein identifying an iteratively 
clustering are performed for search term suggestion (See e.g. Fig. 12 and [0171], "A snapshot of 
the screen displaying five returned text clusters 1216, 1218, 1220, 1222, and 1224 is shown in 
the left half of FIG. 12. These clusters are the clusters closest to the query terms. The user 
decides to scatter the first text cluster containing the terms 'ancient' and 'cathedral' again based 
on text... As described above, this causes the system to refine the existing selected cluster into 
smaller separate clusters"). 

28. The computer-readable medium of claim 22, wherein iteratively clustering further 
comprises propagating clustering results of a first iteration to all related data objects of the multi- 
type data objects, at least two of the related data objects being of heterogeneous data type, the 
propagating being used to enhance clustering of respective ones of the multi-type data objects in 
a second iteration of reinforced clustering operations (See e.g. [0152], "Scatter/Gather iteratively 
refines a search by 'scattering' a collection into a small number of clusters, and then a user 
'gathers' clusters of interest for scattering again. The Scatter/Gather method is extended by the 
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invention to extend to a multi-modal, multi-feature method, using both text and image features to 
navigate a collection of documents with text and images"). 

29. The computer-readable medium of claim 22, wherein iteratively clustering further 
comprises determining a similarity between individual ones of the multi-type data objects, the 
similarity being a function of at least one of object content similarity and similarities between 
respective ones of the relationships (See e.g. [0003], "The invention relates to... an efficient 
scheme for assigning data objects in a collection to clusters based on similarities in their contents 
and characteristics"). 

30. The computer-readable medium of claim 22, wherein iteratively clustering further 
comprises merging related ones of the multi-type data objects to reduce feature space 
dimensionality of the related ones (See e.g. [0033], "various document features in different 
modalities are appropriately weighted and combined to form clusters representative of overall 
similarity"). 

31. The computer-readable medium of claim 22, wherein the instructions further 
comprise instructions for mutually reinforcing importance of individual ones of the multi-type 
data objects within an object type and between different object types (See e.g. [0097], "The use 
of token frequency weight and inverse context frequency weight for the embedding employed by 
the invention is consistent with the following intuitive description. Each additional occurrence of 
an element (or word, for example) in a context (e.g., a document) reflects an increased level of 
importance for that element as a descriptive feature"). 

33. The computer-readable medium of claim 22, and further comprising instructions for: 
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responsive to receiving a term from a user, comparing the term with a feature space of 
objects in the reinforced clusters (See e.g. Fig. 12, "Query Words" 1210); 

responsive to comparing, identifying one or more search term suggestions (See e.g. Fig. 
12 and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms"); and 

communicating the search term suggestions to the user (See e.g. Fig. 12, "Text 
Clusters]" 1216, 1218, 1220, 1222, and 1224). 
34. A computing-device comprising: 

identifying means to identify relationships between multi-type data objects, wherein the 
multi-type data objects comprise at least one object of a first type and at least one object of a 
second type different from the first type (See e.g. [0022], "Disparate types of information such as 
text, image features and usage are referred to as 'modalities.' Multi-modal clustering hence is the 
grouping of objects that have data from several modalities associated with them" and [0033], 
"various document features in different modalities are appropriately weighted and combined to 
form clusters representative of overall similarity"); 

iterative clustering means to iteratively cluster the multi-type data objects in view of 
respective ones of the relationships to generate reinforced clusters (See e.g. [0035], "Using the 
system, a user progressively narrows a collection to a small number of elements of interest, 
similar to the Scatter/Gather system developed for text browsing, except the Scatter/Gather 
method is extended hereby to use multi-modal features" where, see [0043], "iterative clustering 
and selection of cluster subsets can help a user identify images of interest"); and 
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wherin the reinforced clusters are for use by a search term suggestion means to respond 
to a search query from a user with terms relevant to the search query (See e.g. Fig. 12 and 
[0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 1222, 
and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the query 
terms. . . The user decides to scatter the first text cluster containing the terms 'ancient 5 and 
'cathedral' again based on text. . . As described above, this causes the system to refine the 
existing selected cluster into smaller separate clusters"). 

35. The computing device of claim 34, wherein the computing device further comprises 
weighting means to weight respective ones of the relationships to indicate an importance to 
associated objects of the multi-type data objects (See e.g. [0033], "various document features in 
different modalities are appropriately weighted and combined to form clusters representative of 
overall similarity"). 

37. The computing device of claim 34, wherein the iterative clustering means further 
comprise aggregating means to propagate data object relationships to related ones of the multi- 
type data objects based on content of the reinforced clusters (See e.g. [0031], "Each modality 
within each document is described herein by an n-dimensional vector, thereby facilitating 
quantitative analysis of the relationships among the documents in the collection" where, see 
[0076], "As illustrated in FIG. 1, each document (for example, an HTML document 1 10) chosen 
from a collection 120 maps to a set of feature vectors 1 12, one for each modality (for example, a 
text vector 1 14 and a URL vector 1 16)"). 

38. The computing device of claim 34, wherein the iterative clustering means further 
comprise determining means to determine a similarity between individual ones of the multi-type 
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data objects, the similarity being a function of at least one of object content similarity and 
similarities between respective ones of the relationships (See e.g. [0003], "The invention relates 
to... an efficient scheme for assigning data objects in a collection to clusters based on similarities 
in their contents and characteristics"). 

39. The computing device of claim 34, wherein the iterative clustering means further 
comprise merging means to combine related ones of the multi-type data objects to reduce feature 
space dimensionality of the related ones (See e.g. [0033], "various document features in different 
modalities are appropriately weighted and combined to form clusters representative of 

overall similarity"). 

40. The computing device of claim 34, and further comprising: 

comparing means, responsive to receive a term from a user, to compare the term with a 
feature space of objects in the reinforced clusters (See e.g. Fig. 12, "Query Words" 1210); and 

responsive to comparing, identifying one or more search term suggestions (See e.g. Fig. 
12 and [0171], "A snapshot of the screen displaying five returned text clusters 1216, 1218, 1220, 
1222, and 1224 is shown in the left half of FIG. 12. These clusters are the clusters closest to the 
query terms"). 



Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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Claim 36 is rejected under 35 U.S.C. 103(a) as being unpatentable over Schuetze et al., in 
view of Bowman et al. 5 U.S. Pat. 6 5 169,986. 

36. The computing device of claim 34, wherein the computing device further comprises 
determining means to locate a search term suggestion from the reinforced clusters responsive to 
receipt of a bid term, the search term suggestion substantially matching or being related to one or 
more of the multi-type data objects (Schuetze et al. do not teach suggesting search terms in 
response to a bid term. However, Bowman et al. do, see col. 4, lines 34-52. Thus, it would have 
been obvious to one of ordinary skill in the database searching art at the time of the invention to 
combine the teachings of the cited references because Bowman's et al. teachings would have 
allowed Schuetze's et al. method and system to gain the ability "to further refine the query and 
narrow the query result by selecting one or more related query terms that more accurately reflect 
the user's intended request", see col. 1, line 48 - col. 2, line 3). 

Allowable Subject Matter 

Claims 1 1 and 32 are objected to as being dependent upon a rejected base claim, but 
would be allowable if rewritten in independent form including all of the limitations of the base 
claim and any intervening claims. 

Response to Arguments 

As per Applicant's argument that Schuetze et al. does not disclose the limitation 
"identifying relationships between multi-type data objects, wherein the multi-type data objects 
comprise at least one object of a first type and at least one object of a second type different from 
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the first type" in claim 1, the Examiner respectfully disagrees. The Examiner cited Schuetze et al. 
[0022], "Disparate types of information such as text, image features and usage are referred to as 
'modalities.' Multi-modal clustering hence is the grouping of objects that have data from several 
modalities associated with them". While this reference is primarily a definition of terms, it is 
defining the multi-type object clustering disclosed by Schuetze's et al. This identification of and 
relationship among multi-type data objects is further explained in Schuetze et al. [0033], "In an 
alternative application of the invention, various document features in different modalities are 
appropriately weighted and combined to form clusters representative of overall similarity". 

Applicant cited Schuetze et al. Fig. 3 as an example of how Schuetze et al. do not teach 
the claimed invention. However, the Examiner points out that Schuetze et al. describe several 
embodiments of their method, and the Examiner does not refer to the embodiment shown in Fig. 
3 in the rejection of the instant claim. 

As per Applicant's argument that Schuetze et al. does not disclose the limitation 
"iteratively clustering the multi-type data objects in view of respective ones of the relationships 
to generate reinforced clusters" in claim 1 , the Examiner respectfully disagrees. The Applicant 
has not explicitly defined the term "reinforced clusters", thus the Examiner is free to give the 
term its broadest reasonable interpretation in the art. The Examiner cited Schuetze et al. [0043], 
"iterative clustering and selection of cluster subsets can help a user identify images of interest". 
This is further explained in Schuetze et al. [0035], "Using the system, a user progressively 
narrows a collection to a small number of elements of interest, similar to the Scatter/Gather 
system developed for text browsing, except the Scatter/Gather method is extended hereby to use 
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multi-modal features". These references show that the multi-modal clusters are being iteratively 
narrowed into "cluster subsets", or the claimed "reinforced clusters". 

As per Applicant's argument that claims 6, 16, and 27 are not obvious over Scheutze et 
al. in view of Bowman et al. because Schuetze et al. does not teach each limitation of the 
independent claims upon which the instant claims depend, the Examiner disagrees and has 
clarified the rejections of claims 1,13, and 22. 

As per Applicant's argument that the 35 U.S.C. 103(a) rejections of claims 12, 21, 33, 
and 40 admit that Schuetze et al. do not teach each limitation of the independent claims, the 
Examiner respectfully disagrees, and after reviewing the rejections does not see how this could 
have been accomplished. Further, after closer study of Schuetze et al., the Examiner believes 
that the reference does indeed teach suggesting search terms, as depicted in Fig. 12, and has 
withdrawn the 35 U.S.C. 103(a) rejections accordingly. 

Applicant has not argued the 35 U.S.C. 103(a) rejection of claim 36. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure: Pedersen et al., U.S. Pat. 5,442,778 and Heckerman et al., U.S. Pat. 6,742,003. 

Any inquiry concerning this communication or earlier communications from the 
Examiner should be directed to Aaron J. Sanders whose telephone number is 571-270-1016. The 
Examiner can normally be reached on M-Th 8:00a-5:00p. 
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If attempts to reach the Examiner by telephone are unsuccessful, the Examiner's 
supervisor, Tim Vo can be reached on 571-272-3642. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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